Read FIFO Scheduling for Multiple Streams While Maintaining Coherency

ABSTRACT

A method of scheduling trace packets in an integrated circuit generating trace packets of plural types stores trace data in respective first-in-first-out buffers. If a timing trace data first-in-first-out buffer is empty, timing trace data packet is transmitted. If a program counter overall data first-in-first-out buffer is not empty and the processor is at a data interruptible boundary, transmitting a program counter data packet. If data first-in-first-out buffer is not empty, transmitting a data packet. The program counter data packets include program counter sync data, program counter exception data, program counter relative branch data and program counter absolute branch data.

TECHNICAL FIELD OF THE INVENTION

The technical field of this invention is emulation hardware particularly for highly integrated digital signal processing systems.

BACKGROUND OF THE INVENTION

Advanced wafer lithography and surface-mount packaging technology are integrating increasingly complex functions at both the silicon and printed circuit board level of electronic design. Diminished physical access to circuits for test and emulation is an unfortunate consequence of denser designs and shrinking interconnect pitch. Designed-in testability is needed so the finished product is both controllable and observable during test and debug. Any manufacturing defect is preferably detectable during final test before a product is shipped. This basic necessity is difficult to achieve for complex designs without taking testability into account in the logic design phase so automatic test equipment can test the product.

In addition to testing for functionality and for manufacturing defects, application software development requires a similar level of simulation, observability and controllability in the system or sub-system design phase. The emulation phase of design should ensure that a system of one or more ICs (integrated circuits) functions correctly in the end equipment or application when linked with the system software. With the increasing use of ICs in the automotive industry, telecommunications, defense systems, and life support systems, thorough testing and extensive real-time debug becomes a critical need.

Functional testing, where the designer generates test vectors to ensure conformance to specification, still remains a widely used test methodology. For very large systems this method proves inadequate in providing a high level of detectable fault coverage. Automatically generated test patterns are desirable for full testability, and controllability and observability. These are key goals that span the full hierarchy of test from the system level to the transistor level.

Another problem in large designs is the long time and substantial expense involved in design for test. It would be desirable to have testability circuitry, system and methods that are consistent with a concept of design-for-reusability. In this way, subsequent devices and systems can have a low marginal design cost for testability, simulation and emulation by reusing the testability, simulation and emulation circuitry, systems and methods that are implemented in an initial device. Without a proactive testability, simulation and emulation plan, a large amount of subsequent design time would be expended on test pattern creation and upgrading.

Even if a significant investment were made to design a module to be reusable and to fully create and grade its test patterns, subsequent use of a module may bury it in application specific logic. This would make its access difficult or impossible. Consequently, it is desirable to avoid this pitfall.

The advances of IC design are accompanied by decreased internal visibility and control, reduced fault coverage and reduced ability to toggle states, more test development and verification problems, increased complexity of design simulation and continually increasing cost of CAD (computer aided design) tools. In the board design the side effects include decreased register visibility and control, complicated debug and simulation in design verification, loss of conventional emulation due to loss of physical access by packaging many circuits in one package, increased routing complexity on the board, increased costs of design tools, mixed-mode packaging, and design for produceability. In application development, some side effects are decreased visibility of states, high speed emulation difficulties, scaled time simulation, increased debugging complexity, and increased costs of emulators. Production side effects involve decreased visibility and control, complications in test vectors and models, increased test complexity, mixed-mode packaging, continually increasing costs of automatic test equipment and tighter tolerances.

Emulation technology utilizing scan based emulation and multiprocessing debug was introduced more than 10 years ago. In 1988, the change from conventional in circuit emulation to scan based emulation was motivated by design cycle time pressures and newly available space for on-chip emulation. Design cycle time pressure was created by three factors. Higher integration levels, such as increased use of on-chip memory, demand more design time. Increasing clock rates mean that emulation support logic causes increased electrical intrusiveness. More sophisticated packaging causes emulator connectivity issues. Today these same factors, with new twists, are challenging the ability of a scan based emulator to deliver the system debug facilities needed by today's complex, higher clock rate, highly integrated designs. The resulting systems are smaller, faster, and cheaper. They have higher performance and footprints that are increasingly dense. Each of these positive system trends adversely affects the observation of system activity, the key enabler for rapid system development. The effect is called “vanishing visibility.”

FIG. 1 illustrates the trend in visibility and control over time and greater system integration. Application developers prefer the optimum visibility level illustrated in FIG. 1. This optimum visibility level provides visibility and control of all relevant system activity. The steady progression of integration levels and increases in clock rates steadily decrease the actual visibility and control available over time. These forces create a visibility and control gap, the difference between the optimum visibility and control level and the actual level available. Over time, this gap will widen. Application development tool vendors are striving to minimize the gap growth rate. Development tools software and associated hardware components must do more with less resources and in different ways. Tackling this ease of use challenge is amplified by these forces.

With today's highly integrated System-On-a-Chip (SOC) technology, the visibility and control gap has widened dramatically over time. Traditional debug options such as logic analyzers and partitioned prototype systems are unable to keep pace with the integration levels and ever increasing clock rates of today's systems. As integration levels increase, system buses connecting numerous subsystem components move on chip, denying traditional logic analyzers access to these buses. With limited or no significant bus visibility, tools like logic analyzers cannot be used to view system activity or provide the trigger mechanisms needed to control the system under development. A loss of control accompanies this loss in visibility, as it is difficult to control things that are not accessible.

To combat this trend, system designers have worked to keep these buses exposed. Thus the system components were built in a way that enabled the construction of prototyping systems with exposed buses. This approach is also under siege from the ever-increasing march of system clock rates. As the central processing unit (CPU) clock rates increase, chip to chip interface speeds are not keeping pace. Developers find that a partitioned system's performance does not keep pace with its integrated counterpart, due to interface wait states added to compensate for lagging chip to chip communication rates. At some point, this performance degradation reaches intolerable levels and the partitioned prototype system is no longer a viable debug option. In the current era production devices must serve as the platform for application development.

Increasing CPU clock rates are also limiting availability of other simple visibility mechanisms. Since the CPU clock rates can exceed the maximum I/O state rates, visibility ports exporting information in native form can no longer keep up with the CPU. On-chip subsystems are also operated at clock rates that are slower than the CPU clock rate. This approach may be used to simplify system design and reduce power consumption. These developments mean simple visibility ports can no longer be counted on to deliver a clear view of CPU activity. As visibility and control diminish, the development tools used to develop the application become less productive. The tools also appear harder to use due to the increasing tool complexity required to maintain visibility and control. The visibility, control, and ease of use issues created by systems-on-a-chip tend to lengthen product development cycles.

Even as the integration trends present developers with a tough debug environment, they also present hope that new approaches to debug problems will emerge. The increased densities and clock rates that create development cycle time pressures also create opportunities to solve them. On-chip, debug facilities are more affordable than ever before. As high speed, high performance chips are increasingly dominated by very large memory structures, the system cost associated with the random logic accompanying the CPU and memory subsystems is dropping as a percentage of total system cost. The incremental cost of several thousand gates is at an all time low. Circuits of this size may in some cases be tucked into a corner of today's chip designs. The incremental cost per pin in today's high density packages has also dropped. This makes it easy to allocate more pins for debug. The combination of affordable gates and pins enables the deployment of new, on-chip emulation facilities needed to address the challenges created by systems-on-a-chip.

When production devices also serve as the application debug platform, they must provide sufficient debug capabilities to support time to market objectives. Since the debugging requirements vary with different applications, it is highly desirable to be able to adjust the on-chip debug facilities to balance time to market and cost needs. Since these on-chip capabilities affect the chip's recurring cost, the scalability of any solution is of primary importance. “Pay only for what you need” should be the guiding principle for on-chip tools deployment. In this new paradigm, the system architect may also specify the on-chip debug facilities along with the remainder of functionality, balancing chip cost constraints and the debug needs of the product development team.

FIG. 2 illustrates an emulator system 100 including four emulator components. These four components are: a debugger application program 110; a host computer 120; an emulation controller 130; and on-chip debug facilities 140. FIG. 2 illustrates the connections of these components. Host computer 120 is connected to an emulation controller 130 external to host 120. Emulation controller 130 is also connected to target system 140. The user preferably controls the target application on target system 140 through debugger application program 110.

Host computer 120 is generally a personal computer. Host computer 120 provides access the debug capabilities through emulator controller 130. Debugger application program 110 presents the debug capabilities in a user-friendly form via host computer 120. The debug resources are allocated by debug application program 110 on an as needed basis, relieving the user of this burden. Source level debug utilizes the debug resources, hiding their complexity from the user. Debugger application program 110 together with the on-chip trace and triggering facilities provide a means to select, record, and display chip activity of interest. Trace displays are automatically correlated to the source code that generated the trace log. The emulator provides both the debug control and trace recording function.

The debug facilities are preferably programmed using standard emulator debug accesses through a JTAG or similar serial debug interface. Since pins are at a premium, the preferred embodiment of the invention provides for the sharing of the debug pin pool by trace, trigger, and other debug functions with a small increment in silicon cost. Fixed pin formats may also be supported. When the pin sharing option is deployed, the debug pin utilization is determined at the beginning of each debug session before target system 140 is directed to run the application program. This maximizes the trace export bandwidth. Trace bandwidth is maximized by allocating the maximum number of pins to trace.

The debug capability and building blocks within a system may vary. Debugger application program 100 therefore establishes the configuration at runtime. This approach requires the hardware blocks to meet a set of constraints dealing with configuration and register organization. Other components provide a hardware search capability designed to locate the blocks and other peripherals in the system memory map. Debugger application program 110 uses a search facility to locate the resources. The address where the modules are located and a type ID uniquely identifies each block found. Once the IDs are found, a design database may be used to ascertain the exact configuration and all system inputs and outputs.

Host computer 120 generally includes at least 64 Mbytes of memory and is capable of running Windows 95, SR-2, Windows NT, or later versions of Windows. Host computer 120 must support one of the communications interfaces required by the emulator. These may include: Ethernet 10T and 100T, TCP/IP protocol; Universal Serial Bus (USB); Firewire IEEE 1394; and parallel port such as SPP, EPP and ECP.

Host computer 120 plays a major role in determining the real-time data exchange bandwidth. First, the host to emulator communication plays a major role in defining the maximum sustained real-time data exchange bandwidth because emulator controller 130 must empty its receive real-time data exchange buffers as fast as they are filled. Secondly, host computer 120 originating or receiving the real-time data exchange data must have sufficient processing capacity or disc bandwidth to sustain the preparation and transmission or processing and storing of the received real-time data exchange data. A state of the art personal computer with a Firewire communication channel (IEEE 1394) is preferred to obtain the highest real-time data exchange bandwidth. This bandwidth can be as much as ten times greater performance than other communication options.

Emulation controller 130 provides a bridge between host computer 120 and target system 140. Emulation controller 130 handles all debug information passed between debugger application program 110 running on host computer 120 and a target application executing on target system 140. A presently preferred minimum emulator configuration supports all of the following capabilities: real-time emulation; real-time data exchange; trace; and advanced analysis.

Emulation controller 130 preferably accesses real-time emulation capabilities such as execution control, memory, and register access via a 3, 4, or 5 bit scan based interface. Real-time data exchange capabilities can be accessed by scan or by using three higher bandwidth real-time data exchange formats that use direct target to emulator connections other than scan. The input and output triggers allow other system components to signal the chip with debug events and vice-versa. Bit I/O allows the emulator to stimulate or monitor system inputs and outputs. Bit I/O can be used to support factory test and other low bandwidth, non-time-critical emulator/target operations. Extended operating modes are used to specify device test and emulation operating modes. Emulator controller 130 is partitioned into communication and emulation sections. The communication section supports host communication links while the emulation section interfaces to the target, managing target debug functions and the device debug port. Emulation controller 130 communicates with host computer 120 using one of industry standard communication links outlined earlier herein. The host to emulator connection is established with off the shelf cabling technology. Host to emulator separation is governed by the standards applied to the interface used.

Emulation controller 130 communicates with the target system 140 through a target cable or cables. Debug, trace, triggers, and real-time data exchange capabilities share the target cable, and in some cases, the same device pins. More than one target cable may be required when the target system 140 deploys a trace width that cannot be accommodated in a single cable. All trace, real-time data exchange, and debug communication occurs over this link. Emulator controller 130 preferably allows for a target to emulator separation of at least two feet. This emulation technology is capable of test clock rates up to 50 MHZ and trace clock rates from 200 to 300 MHZ, or higher. Even though the emulator design uses techniques that should relax target system 140 constraints, signaling between emulator controller 130 and target system 140 at these rates requires design diligence. This emulation technology may impose restrictions on the placement of chip debug pins, board layout, and requires precise pin timings. On-chip pin macros are provided to assist in meeting timing constraints.

The on-chip debug facilities offer the developer a rich set of development capability in a two tiered, scalable approach. The first tier delivers functionality utilizing the real-time emulation capability built into a CPU's mega-modules. This real-time emulation capability has fixed functionality and is permanently part of the CPU while the high performance real-time data exchange, advanced analysis, and trace functions are added outside of the core in most cases. The capabilities are individually selected for addition to a chip. The addition of emulation peripherals to the system design creates the second tier functionality. A cost-effective library of emulation peripherals contains the building blocks to create systems and permits the construction of advanced analysis, high performance real-time data exchange, and trace capabilities. In the preferred embodiment five standard debug configurations are offered, although custom configurations are also supported. The specific configurations are covered later herein.

SUMMARY OF THE INVENTION

Emulation trace generates data streams used to trace processor activity. When multiple data streams are employed, they are written at different times into individual first-in-first-out (FIFO) buffers. Export of these data streams may have differing priority levels. The trace mechanism must have some manner to order the separate data streams to preserve coherency and the usefulness of the trace data.

This invention enables real time tracing of processor activity generating plural trace data streams. These trace data streams enable the user to analyze and debug application programs under development. This invention preserves coherency of the plural data streams to enhance the usefulness of the trace data.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of this invention are illustrated in the drawings, in which:

FIG. 1 illustrates the visibility and control of typical integrated circuits as a function of time due to increasing system integration;

FIG. 2 illustrates an emulation system to which this invention is applicable;

FIG. 3 illustrates in block diagram form a typical integrated circuit employing configurable emulation capability;

FIG. 4 illustrates in block diagram form a detail of trace subsystem;

FIG. 5 illustrates in block diagram form a detail of the trace packet merge block of FIG. 4;

FIG. 6 illustrates the location of interruptible boundaries according to this invention; and

FIG. 7 illustrates in flow chart form the process of this invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Data streams are generated for tracing target processor activity. When multiple streams are on, they are written at different times into their individual FIFO. Their export can have different priority levels. However the data coming out should be in order so that the user does not lose coherency.

When the streams are switched on or off using the triggers, markers are generated. Various streams are synchronized using markers called sync points. The sync points provide a unique identifier field and a context to the data that will follow it. All streams may generate a sync point with this unique identifier. The information in the sync point is valid only at a legal instruction boundary.

FIG. 3 illustrates an example of one on-chip debug architecture embodying target system 140. The architecture uses several module classes to create the debug function. One of these classes is event detectors including bus event detectors 210, auxiliary event detectors 211 and counters/state machines 213. A second class of modules is trigger generators including trigger builders 220. A third class of modules is data acquisition including trace collection 230 and formatting. A fourth class of modules is data export including trace export 240, and real-time data exchange export 241. Trace export 240 is controlled by clock signals from local oscillator 245. Local oscillator 245 will be described in detail below. A final class of modules is scan adaptor 250, which interfaces scan input/output to CPU core 201. Final data formatting and pin selection occurs in pin manager and pin micros 260.

The size of the debug function and its associated capabilities for any particular embodiment of a system-on-chip may be adjusted by either deleting complete functions or limiting the number of event detectors and trigger builders deployed. Additionally, the trace function can be incrementally increased from program counter trace only to program counter and data trace along with ASIC and CPU generated data. The real-time data exchange function may also be optionally deployed. The ability to customize on-chip tools changes the application development paradigm. Historically, all chip designs with a given CPU core were limited to a fixed set of debug capability. Now, an optimized debug capability is available for each chip design. This paradigm change gives system architects the tools needed to manage product development risk at an affordable cost. Note that the same CPU core may be used with differing peripherals with differing pin outs to embody differing system-on-chip products. These differing embodiments may require differing debug and emulation resources. The modularity of this invention permits each such embodiment to include only the necessary debug and emulation resources for the particular system-on-chip application.

The real-time emulation debug infrastructure component is used to tackle basic debug and instrumentation operations related to application development. It contains all execution control and register visibility capabilities and a minimal set of real-time data exchange and analysis such as breakpoint and watchpoint capabilities. These debug operations use on-chip hardware facilities to control the execution of the application and gain access to registers and memory. Some of the debug operations which may be supported by real-time emulation are: setting a software breakpoint and observing the machine state at that point; single step code advance to observe exact instruction by instruction decision making; detecting a spurious write to a known memory location; and viewing and changing memory and peripheral registers.

Real-time emulation facilities are incorporated into a CPU mega-module and are woven into the fabric of CPU core 201. This assures designs using CPU core 201 have sufficient debug facilities to support debugger application program 110 baseline debug, instrumentation, and data transfer capabilities. Each CPU core 201 incorporates a baseline set of emulation capabilities. These capabilities include but are not limited to: execution control such as run, single instruction step, halt and free run; displaying and modifying registers and memory; breakpoints including software and minimal hardware program breakpoints; and watchpoints including minimal hardware data breakpoints.

Consider the case of tracing processor activity and generating timing, program counter and data streams. Table 1 shows the streams generated when a sync point is generated. Context information is provided only in the program counter stream. There is no order dependency of the various streams with each other except that the sync point identifiers cannot exceed each other by more than 7. Therefore, if program counter stream has yet to send out program counter sync point of id=1, then timing stream could have sent out it's sync points with ids from 1 to 7. The timing stream cannot send out it's next sync point of 1. Within each stream the order cannot be changed between sync points. TABLE 1 Timing stream PC stream Data stream Timing sync point, PC sync point, Data sync point, id = 1 id = 1 id = 1 Timing data PC data Memory Data Timing data Memory Data Timing data PC data Memory Data PC data Timing data Memory Data Timing sync point, PC sync point, Data sync point, id = 2 id = 2 id = 2

The program counter stream is further classified into exceptions, relative branches, absolute branches and sync points. Since the data size for each of these sub-streams is different there is a separate FIFO for each of them. There are the following FIFOs:

1. A timing FIFO has both timing data and timing sync points.

2. A program counter sync point FIFO keeps track of the program counter sync points in the program counter stream.

3. An exceptions FIFO keeps track of all exception information in the program counter stream.

4. An absolute branches FIFO keeps track of the registered branches in the program counter stream.

5. A relative branches FIFO keeps track of the relative branch packets in the program counter stream.

6. A program counter overall FIFO (val FIFO) keeps track of the order of program counter data (program counter sync points, exceptions, absolute branches, relative branches) being written in the various program counter streams.

7. An memory FIFO has both data logs and data sync points.

FIG. 4 illustrates a detail of the trace subsystem. Data to be traced is generated by central processing unit core 201, peripheral and memory system 202, the central processing unit mailbox 203 and application specific integrated circuit (ASIC) data source 204. Central processing unit core 201 generates program counter and timing data. Peripherals and memory system 202 generates memory read and write access addresses and corresponding data. Central processing unit mailbox 203 generates data handled by application programs. ASIC data source 204 generates data from special purpose hardware particular to that integrated circuit. These trace data sources supply data to trace collection subunit 233. Trace collection subunit 233 includes separate sections for receipt of program counter/timing data from central processing unit core 201 (sections 510/520), memory read and writes from peripheral and memory system 202 (sections 530/540), application program data from central processing unit mailbox 203 (section 550) and ASIC data from ASIC data source 204 (section 560). Trace collection subunit 233 also receives triggering signals from trigger builders 220. Trace collection subunit 233 produces plural separate data streams corresponding to the received trace data.

Trace packet merge unit 237 receives the plural trace data streams from trace collection subunit 233. Trace packet merge unit 237 merges these plural data streams into a single trace data stream. Trace packet merge unit 237 supplies this merged trace data stream to trace export 240. Trace export 240 drives pin manager 260 under timing control of export clock generator 245 (corresponding to local oscillator 245 illustrated in FIG. 3).

FIG. 5 illustrates in block diagram form a detail of the trace packet merge unit 237. FIG. 5 illustrates six separate trace data streams that must be merged for output. Timing information 510 comes from central processing unit core 201, has a steady rate and a high average bandwidth. It is typically output without compression. Trace packet merge unit 237 gives this data the highest priority. Because of its high priority, trace information 510 can be buffered in first-in-first-out (FIFO) buffer 511 with a small capacity. The high priority ensures that the small FIFO buffer 511 will not overflow.

Program counter flow information 520 also comes from central processing unit core 210. Program counter flow information 520 follows the path of program execution of central processing unit core 201. Program counter flow information has a variable rate and a medium average bandwidth. It is typically transmitted with high compression. Trace packet merge unit 237 gives this data the second highest priority just lower than timing information. The medium bandwidth and high compression require a small FIFO buffer 521.

Memory write information 530 comes from peripherals and memory system 202. The information flow occurs primarily in bursts of high activity interspersed with times of low activity. This memory write information 530 is typically transmitted with low compression. A high bandwidth is required to accommodate the bursts. Trace packet merge unit 237 gives this data an intermediate priority just lower than program counter flow information 520. The irregular rate and low compression require a large FIFO buffer 531.

Memory read information 540 also comes from peripherals and memory system 202. Memory read information 540 is similar to memory write information 530. Memory read information 540 occurs primarily in bursts of high activity interspersed with times of low activity and is generally transmitted with low compression. A high bandwidth is required to accommodate the bursts. Trace packet merge unit 237 gives this data an intermediate priority just lower than memory write information 530. The irregular rate and low compression require a large FIFO buffer 541.

Application supplied data 550 comes from central processing unit mailbox 203. It is difficult to characterize this data because it varies depending on the application program running on central processing unit 201. Application supplied data 550 is generally believed to occur in bursts with a low average bandwidth. This data is typically transmitted with low compression. Application supplied data 550 has a low priority in trace packet merge unit 237 below that of the memory write information 530 and memory read information 540. The particular application program determines the size required of FIFO buffer 551.

ASIC supplied data 560 comes from (ASIC) data source 204. The data rate, required bandwidth and required size of FIFO buffer 561 depend on the particular integrated circuit and can't be generalized. Trace packet merge unit 237 gives this data the lowest priority.

Scheduler 570 receives data from the six FIFO buffers 511, 521, 531, 541, 551 and 561. Scheduler 570 merges these separate data streams into a single trace packet stream for export via the debug port (FIGS. 3 and 4).

The timing stream gets the highest priority on the read side as long as the timing and program counter data stay in the range of seven sync points. The program counter stream gets the next highest priority and the data stream gets the lowest priority. Timing packets can be sent out at any time even though there may be incomplete program counter or memory packets. Sending memory packets can be intervened by a program counter packet provided the data log is at an interruptible boundary. An interruptible boundary for data trace is the boundary after sending out the entire data value and before the data address or program counter address.

FIG. 6 illustrates boundary 601 between data value and data address and boundary 602 between data address and program counter address.

FIG. 7 illustrates process 700 which depicts the read priority for the various FIFOs. Process 700 begins with master read block 701 which determines the full or empty states of the various FIFOs. At decision block 702 process 700 tests to determine if the timing FIFO is empty (decision block 702). If the timing FIFO is not empty (No at decision block 702), then process 700 sends a timing packet (processing block 703). Control then returns to master read block 701.

If the timing FIFO is empty (Yes at decision block 702), then process 700 tests to determine if the val FIFO is empty (decision block 704). The program counter overall FIFO (val FIFO) keeps track of the order of program counter data (program counter sync points, exceptions, absolute branches, relative branches) being written in the various program counter streams. If the val FIFO is not empty (No at decision block 704), then process 700 tests to determine if the process is at a data interruptible boundary (decision block 705). These data interruptible boundaries 601 and 602 are illustrated in FIG. 6. If the process is not at a data interruptible boundary (No at decision block 705), then process 700 sends a data packet (processing block 706). Process 700 then returns to master read block 701.

If the process is a an interruptible boundary empty (Yes at decision block 705), then process 700 performs a series of tests to determine the type of program counter data stored. If this program counter data is program counter sync data (Yes at decision block 707), then process 700 sends a program counter sync packet (processing block 708). If this program counter data is program counter exception data (Yes at decision block 709), then process 700 sends a program counter exception packet (processing block 710). If this program counter data is program counter relative branch data (Yes at decision block 711), then process 700 sends a program counter relative branch packet (processing block 712). If this program counter data is program counter absolute branch data (Yes at decision block 713), then process 700 sends a program counter relative branch packet (processing block 714). Process 700 returns to master read block 701 after sending any of these packets.

If the val FIFO was empty (Yes at decision block 715), then process 700 tests to determine if the data FIFO is empty (decision block 715). If the data FIFO is empty (Yes at decision block 715), then process 700 returns to master read block 701. All the FIFOs are empty, so no action is needed. If the data FIFO is not empty (Yes at decision block 715)m then process 700 sends a data packet (processing block 706). Process 700 then returns to master read block 701. 

1-4. (canceled)
 5. A method of scheduling trace packets in an integrated circuit generating trace packets of plural types having a predetermined hierarchy of trace data types from a highest priority to a lowest priority, the method comprising the steps of: storing trace data of the plural types in respective first-in-first-out buffers; and for each trace data type from a highest priority to a lowest priority: determining if a first-in-first-out buffer for trace data having a current priority is empty, if the first-in-first-out buffer for trace data having the current priority is not empty, then transmitting a trace data packet of trace data of the current priority, and if the first-in-first-out buffer for trace data having the current priority is empty, then setting the current trace data type to the trace data type of a next lower priority, until trace data types of all priorities in said hierarchy are considered.
 6. The method of scheduling trace packets of claim 5, wherein: at least one of said plural trace data types includes a plurality of trace data subtypes having a predetermined hierarchy of trace data subtypes from a highest priority to a lowest priority; and said step of transmitting a trace data packet of said at least one trace data type if the corresponding first-in-first-out buffer is not empty includes transmitting trace data packets in priority order from the highest priority trace data subtype to the lowest priority trace data subtype.
 7. A trace apparatus scheduling trace packets in an integrated circuit generating trace packets of plural types having a predetermined hierarchy of trace data types from a highest priority to a lowest priority comprising: a plurality of first-in-first-out buffers having an input receiving trace data of a corresponding trace data type and an output; and a trace packet scheduler having a plurality of inputs connected to said outputs of respective first-in-first-out buffers and an output, said trace packet scheduler operative for each trace data type from a highest priority to a lowest priority to determine if said first-in-first-out buffer corresponding to a current trace data type is empty, if said first-in-first-out buffer corresponding to said current trace data type is not empty, then transmit a trace data packet of trace data of the current trace data type, and if said first-in-first-out buffer corresponding to said current trace data type is empty, then setting the current trace data type to the trace data type of a next lower priority, until trace data types of all priorities in said hierarchy are considered.
 8. The trace apparatus of claim 7, wherein: at least one of said plural trace data types includes a plurality of trace data subtypes having a predetermined hierarchy of trace data subtypes from a highest priority to a lowest priority; and said trace packet scheduler is operative to transmit a trace data packet of said at least one trace data type if the corresponding first-in-first-out buffer is not empty including transmitting trace data packets in priority order from the highest priority trace data subtype to the lowest priority trace data subtype. 